-
Notifications
You must be signed in to change notification settings - Fork 303
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce SqliteObserver and use it to collect stats related to queries and db size #2707
Conversation
1799527
to
153a9a2
Compare
153a9a2
to
aab4f76
Compare
aab4f76
to
ee5e13c
Compare
ce7963b
to
5dee08b
Compare
5dee08b
to
5e2e412
Compare
@justin-mp The new changes are the ones from here |
5e2e412
to
0ea3375
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The makeActorCache
changes seem reasonable to me.
@@ -1096,6 +1105,8 @@ void SqliteDatabase::Query::nextRow() { | |||
db.currentRegulator = regulator; | |||
|
|||
int err = sqlite3_step(statement); | |||
rowsRead = getRowsRead(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems inefficient as we're querying SQLite for rows read/written counts on every single row just in case the database is later resent before the query is destroyed -- which is very rare.
Instead, how about putting code into Query::beforeSqliteReset()
that posts the metrics early (calls addQueryStats
directly), and then have the destructor only post metrics if it hasn't been reset (i.e. maybeStatement
is non-null).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I did try that approach first. The problem with that comes in the way we do getRowsRead() and the way the Statement and Query are setup. There are a couple of corner cases that have to be handled - when the Query has its own statement vs passed in a statement, handling the beforeSqliteReset in both Query and Statement.
In the end, I decided to go this route as it is easy to reason about.
As far as inefficiency in querying Sqlite for rows read/written, looking at the implementation of getRowsRead/Written, it does a sqlite3_stmt_status(...)
which does a pointer read, so its not as inefficient as we think it is
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a todo with more details
0ea3375
to
4b48d70
Compare
4b48d70
to
222d244
Compare
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
…bserver Introduce SqliteObserver and use it to collect stats related to queries and db size
https://jira.cfdata.org/browse/STOR-3710